COMMENT ⊗   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	∂23-APR-75  1642		network site SRI
C00059 ENDMK
C⊗;
∂23-APR-75  1642		network site SRI
 Date: 22 APR 1975 1722-PDT
 From: GARVEY at SRI-AI
 Subject: CHAPTER 2
 To:   TOB at SU-AI
 
 Tom, here is chapter 2 (revised). It includes an expansion
 of what was previously chapter 2, and also what was to be chapter 3.
 I will send the figures via inter-office mail. The figures that are
 taken from the display will be replaced by new ones taken from the RAMTEK.
 
 
 
 
 
 
 
 
 
 
 Chapter	2	ISIS and Interactive Strategy Generation
 
 
 I   Introduction
 
 	We have chosen as our objective the task of finding
 specified objects in relatively complex real-world scenes.  Ideally,
 one would like to program a computer to find an unfamiliar object as
 one would instruct a person, by providing a crude description of the
 desired object.  A tabletop, for example, might be described as a
 "horizontal, buff-colored, planar region, 2-1/2 feet above the
 floor."  The computer could demonstrate comprehension by outlining
 instances of the described object in a displayed image.  The programmer
 could then refine the description empirically to correct errors in the
 computer's interpretation.
 
 	Communications on this level with a machine is hampered because
 the machine does not share an "understanding" with the human of basic
 descriptive concepts like "horizontal," "buff," "above," and
 "planar."  Such concepts are often difficult to express in natural
 language, much less in conventional programming languages; primitive
 pictorial concepts are better conveyed in terms of pictorial examples.
 
 	The difficulties incurred in the symbolic description of pictorial
 concepts suggest a pragmatic approach to representation.  Although we
 may not know, in general, how to describe shapes and textures, such
 concepts can often be adequately discriminated by simple feature 
 extraction operators when the context is suitably limited.*REF*
 
 	Representations of this sort are founded solely on empiricism;
 we have no theoretical understanding of perception from which they can
 be deduced.  Our aim has therefore been to create an interactive system
 to facilitate the empirical design of scene analysis programs.  The
 system allows an experimenter to describe basic perceptual concepts
 to the computer using pictorial examples.  The examples are designated
 graphically by encircling portions of a displayed scene with a cursor.
 Given a pictorial example, a representation can be empirically constructed
 by trying pictorial operators in order of increasing complexity until
 the example is sufficiently distinguished from previously determined
 representations.  Pictorially defined concepts constitute a shared
 vocabulary that can be used to describe objects or to define more
 complex concepts.  This paradigm is intended to elevate the user
 above the details of handcoded algorithms, allowing him instead to
 concentrate on the construction of descriptions and strategies.
 
 
 II   Defining Pictorial Representations Interactively
 
 A.	Overview of System
 
 	The system consists basically of an image file, a library of
 primitive extraction functions (see Appendix 2), a means for applying
 selected primitive operators to graphically designated areas of a scene,
 a semantic data structure for accumulating concept definitions, and an
 iconic data structure for retaining pictorial examples of those concepts.
 Images are stored in 120 X 120 sample arrays, each sample characterized
 by TV brightness through red, green, blue, and neutral filters and by a
 range measurement.  (The range measurements are the actual and simulated
 output of a time-of-flight range finder.)  The feature extraction
 operators transform the raw data into more meaningful dimensions such
 as hue, saturation, and local surface orientation.
 
 	The symbolic data structure is a semantic net whose nodes
 represent prototype patterns for objects, attributes, and relations.
 Prototypes are defined symbolically in terms of other nodes or 
 iconically by reference to designated regions of a digitized image.
 The system can obtain values for unspecified properties of a concept
 by applying operators to its examples.  Figure 1(a-c) illustrates the
 conceptual layout for the semantic net.  In Figure 1(c), the object
 tabletop and the attribute buff are defined iconically, while the
 object floor and the attribute horizontal are defined symbolically
 in terms of other nodes.  Floor is distnguished from tabletop by height.
 
 	Iconic regions are represented as an explicit list of picture
 samples and/or as a list of vertices describing a closed polygonal
 boundary.  A bounding rectangle of minimum area is also stored for
 each region.  Efficient routines exist for determining whether a given
 picture element is contained within a bounded region, for obtaining a
 set of samples over a bounded region, and for fitting a boundary around
 a set of samples (see Appendix 2).
 
 	The primitive functions and symbolic data structure reside
 within the interactive environment of INTERLISP 
 on a PDP-10.  The raw data arrays, and support routines for graphics,
 file handling, coordinate transformations and such reside in a FORTRAN
 environment accessible as a lower fork through the TENEX operating system.
 The graphics hardware consists of a monochromatic ADAGE vector display with 
 a "mouse" and a RAMTEK color display with trackball.  
 
 	Briefly, the system's role can be explained as follows.  To
 describe concepts in terms of the computer's primitive functions, a
 human must know (or be able to determine easily) what those primitive
 functions can distinguish.  Our system provides the trainer with tools
 to choose primitive functions empirically.  The trainer can circle regions
 of the displayed image and obtain from the system average or extreme
 numerical values for local operators such as height, hue, saturation,
 and surface normal.  Thus, by applying operators to examples and
 counterexamples of pictorial concepts in the image, the trainer can
 discover directly which operators provide sufficient discrimination.
 The trainer can try out a proposed description by requesting the system
 to illuminate all parts of the displayed scene that correspond to the
 description.
 
 	Let us now illustrate, with portions of actual protocols, how
 an investigator might use the system to define pictorial concepts in a
 simple room scene such as Figure 2.  (The examples presume a slight
 familiarity with LISP notation, but should be largely self explanatory.)
 
 B.	Description of Objects by Pictorial Example
 
 	The trainer's immediate objective in a new domain is to describe
 the principal surfaces of interest (e.g., wall, floor, tabletop in a
 room scene).  We will be working with the displayed scene depicted by 
 Figure 3.  Using the cursor, the trainer begins by outlining 
 the tabletop surface in Figure 3. Tabletop is now a symbolically
 accessible concept, defined by pictorial example.  The trainer can obtain
 primitive features of Tabletop, using the functions in Appendix 2.  For
 instance:
 
 (HEIGHT TABLETOP)	--	user input shown capitalized
 
 2.4			--	machine responds with average height
 				of the encircled surface, shown here
 				in feet.
 
 (HEIGHTEXT TABLETOP)	
 
 (2.43 2.48)		--	(minimum height, maximum height)
 
 (HUE TABLETOP)
 
 31.5			--	average hue over the tabletop,
 				expressed as the angle (in degrees)
 				measured from pure red around the
 				periphery of a color triangle (see
 				Appendix 2).
 
 	The system computes values of surface attributes by applying
 primitive operators to image points randomly sampled over the encircled
 area.  Figure 4 shows the sample points used by the system in computing
 tabletop properties.
 
 	The system can be commanded with the function (SHOWCOLOR TABLETOP)
 to display the color coordinates of the tabletop samples on a standard
 color triangle.  The dense cluster of dots in the center of Figure 5
 corresponds to the color coordinates of the tabletop samples in Figure 4.
 The distance of this cluster from the central white point is a 
 measure of saturation or color purity.  The dashed wedge delineates
 the spectral range of hues observed on the tabletop.  The buff colored
 table samples thus appear in a narrow range of unsaturated yellow (on
 the red-green leg of the color triangle).  The color triangle form of
 presentation is a significant aid in determining whether two colors can
 be distinguished by the computer on the basis of hue or saturation (see
 Appendix 2).
 
 C.	Testing Descriptions
 
 	The trainer uses the information supplied by the system to select
 primitive functions that best distinguish the tabletop from other surfaces
 in the scene.  The adequacy of a set of primitive functions can be
 tested empirically by requesting the system to intensify all points in
 the displayed image that satisfy a proposed description.  More
 specifically, the description is first formulated as a LISP predicate
 using the primitive operators and the predicate functions
 in Appendix 2.  The routine FILTER applies the predicate to randomly
 sampled points in the image.  Those points satisfying the predicate
 will be displayed and retained in order to test subsequent refinements
 to the description.  To illustrate, if the trainer wished to determine
 whether tabletops were adequately distinguished by their color and
 height, he could request the system to filter random image points
 according to the predicate:
 
 			(AND (COLORP (HUEEXT TABLETOP)
 
 					      (HUE X))
 
 			    (LIMITP (HEIGHTEXT TABLETOP)
 
 					  (HEIGHT X)))
 
 This predicate translates roughly as:  "Find all points, X, such that
 the hue and height of X are within the range of colors and heights
 observed on the tabletop."  When the system applied this predicate to
 the random image samples displayed in Figure 6, the display in Figure 7
 was generated.  The left half of this display shows the location in the
 scene of those points from the original set having appropriate height
 and hue.  The corresponding color coordinates of these selected points
 are displayed in the color triangle to the right.  Note that all of
 the selected points fall within the wedge  delimiting the range of hues
 observed in the tabletop example.  Figure 7 shows that while the above
 predicate selected all points on the tabletop, it also accepted some
 points at table height on the buff colored wall and brown door.  The
 definition proposed for the tabletop is thus not sufficiently discriminating
 to select only the tabletop, and must be augmented with a constraint on
 color saturation or on surface orientation.  Figure 8 shows the result
 of further filtering the points of correct height and color with a
 predicate the constrains orientation.
 
 			(LIMITP (ORIENTEXT TABLETOP)
 
 					 (ORIENT X))
 
 	Figure 8 confirms that color, height, and orientation comprise a
 description that distinguishes the tabletop, at least in this particular
 scene.  The corresponding three-clause predicate
 
 		   (AND (COLORP (HUEEXT TABLETOP)(HUE X))
 
 			(LIMITP  (HEIGHTEXT TABLETOP)  (HEIGHT X))
 
 			(LIMITP  (ORIENTEXT TABLETOP)  (ORIENTX)))
 
 expresses the description in procedural form.  This predicate is attached
 to the node for tabletop in the semantic net, as the function definition
 of the atom TABLETOP.  The expression (TABLETOP X) can now be used as a
 predicate to locate tabletop points in other images.  It can also appear
 as a clause in more complex predicates, such as procedures for finding
 tables or objects on tables.
 
 	Since all of the primitive operators appearing in the tabletop
 predicate accept any defined concept as an argument, the predicate can
 be used not only to filter image samples, but also to test whether
 objects or regions qualify as tabletops.  In particular, the trainer can
 have the system apply the predicate (TABLETOP X) to every surface
 description in the data base, in order to determine whether tabletop
 is adequately distinguished from objects that appeared in other scenes.
 
 	In addition to using extreme values of attributes for descriptions,
 it is often useful, when more than a few objects must be distinguished
 from each other, to use descriptions that are more restrictive.  Many 
 objects tend to have overlapping ranges of attribute values.  For example, 
 the saturation over a region of tabletop may vary from .2 to .4 while
 the saturation of wall may vary from .3 to .5.  Obviously, saturation
 alone may not always be adequate to distinguish tabletop samples from
 wall samples.  This becomes a problem when we are trying to
 locate samples of one object or the other in an image where both may 
 be found. A more promising approach is to refine the description in 
 such a way that only a part of the full range of attribute values for the
 object is used to locate initial samples, and then use these initial
 samples as starting points for a region grower. The region grower would
 examine adjacent points to see that they had attribute values within
 the full range of allowable values for the object.
 
 	This partial attribute range is then carefully chosen so as to
 both distinguish the object from other objects, and also be 
 characteristic of the object we are describing.  We may then need
 to augment the description by the addition of other attributes in order
 that the description be truly distinguishing.
 
 	Since the attribute value functions mentioned previously and in
 Appendix 2 only return single composite values for samples and regions, we 
 have included the capability to display histograms of attribute values.
 One-dimensional histograms allow us to examine the 
 complete spectrum of attribute values for an object. In addition, we can
 display two-dimensional histograms (scatter plots) of selected
 attribute pairs (e.g., hue vs. saturation). Points in the histograms
 can be selected and image points with corresponding attribute values 
 can be displayed.  The histograms also allow us to examine 
 windows in the picture (in addition to single points), and check them for
 appropriate statistical distributions of attribute values. This would be
 useful for experimenting with certain classes of texture operators.
 By comparing histograms from the object we are describing with those 
 from other objects, we have been able to interactively determine good 
 distinguishing descriptions for all the objects in our scenes.
 
 
 
 D.	Description of Attributes
 
 	Instead of describing surfaces pictorially, the trainer can first
 define common surface attributes, such as the color "buff" and the
 orientation "horizontal," with which to describe surfaces symbolically.
 These attribute labels can usually be defined by a characteristic range
 of values for a single primitive operator.  The range of values may be
 determined empirically by applying the operator to pictorial examples of
 the attribute.  Horizontal, for instance, might be defined as any
 surface whose normal is within 5 degrees of the Z axis (based on values
 of the ORIENT operator obtained when pointing at image samples on the
 floor and tabletop).  Attribute labels may also be extensionally
 defined in terms of the observable properties of pictorially described
 objects.  For example, buff can be defined for the system as the color
 of the tabletop:
 
 			NEWATTRIBUTE (BUFF)
 
 		defined as:  (HUEEXT TABLETOP)
 
 	Attribute descriptions are tested and refined empirically in
 the same manner as object descriptions.  For instance, the definition of
 horizontal would be tested by requesting the system to intensify points
 randomly sampled in the image that satisfied the predicate:
 
 			(LIMITP HORIZONTAL (ORIENT X))
 
 The symbol HORIZONTAL evaluates to the range of orientation (-5 5).
 Hence the predicate will accept all points, X, with orientations in
 this allowed range.  Figure 9 validates this definition of horizontal,
 since all intensified points are on surfaces such as the floor, tabletop,
 and chairseat normally thought of as horizontal.  This description is
 again preserved in procedural form as the function definition of
 HORIZONTAL.
 
 E.	Symbolic Description of Objects
 
 	Defined attributes can be used symbolically to describe new
 objects.  For example, floor can now be described to the system in
 terms of buff and horizontal as follows:
 
 			NEWOBJECT (FLOOR)
 
 			(ORIENTEXT FLOOR HORIZONTAL)
 
 			(HUEEXT FLOOR BUFF)
 
 			(HEIGHTEXT FLOOR (-0.1 0.2))
 
 NEWOBJECT first establishes a node for floor in the semantic net.  The
 primitive functions then deposit their second argument as corresponding
 property-list values of floor.  The height range (-0.1 0.2) was determined
 empirically by applying the HEIGHT operator to a few manually designated
 floor samples.  Since no pictorial example of floor is provided, any
 attributes not explicitly supplied are indeterminate.  The function
 (BRIGHT FLOOR), for example, would return as a value the atom UNDEF
 (standing for undefined).
 
 	The description of a floor can be tested in the usual way by
 applying the following predicate to randomly sampled image points:
 
 		(AND
 
 			(HORIZONAL X)
 
 			(BUFF X)
 
 			(LIMITP (HEIGHTEXT FLOOR) (HEIGHT X))).
 
 The satisfactory result is displayed in Figure 10.
 
 F.	Description of Complex Objects
 
 	We use the representations developed for simple objects 
 and surfaces to form articulated descriptions of complex objects.
 A chair, for example, might be described at the top level as
 an object with two principal parts, seat and back, in the spatial 
 relationship (ABOVE BACK SEAT). The attributes of seat and back in
 correct spatial juxtaposition adequately distinguish chairs from other
 office furniture (e.g., tables), and may do so more effectively than
 attributes of parts such as legs.  At the next level of description
 the sub-objects, seat and back, might be adequately described in terms
 of their distinguishing surface attributes.  A seat, for instance,
 might be characterized simply as a horizontal surface in a suitable
 height range, perhaps with specified colors and linear dimensions.
 
 	Articulated descriptions can be developed empirically in the
 same way as surface descriptions; the proposed chair description, for
 example, might be tested by writing a program to find chairs, based
 on predicates which find the parts:  e.g.,
 
 		(PROG (X Y)
 
 		    A (COND ((SETQ X (SEAT))
 
 			     (COND ((SETQ Y 
 
 				     (SCAN-ABOVE X (QUOTE BACK)))
 
 				    (RETURN (LIST X Y)))
 
 				   (T (GO A))))
 
 			    (T (RETURN NIL))))
 
 This example function attempts to find the chair by first finding the
 seat, and then looking in a window above the seat region for samples
 that look like the back.  If it finds both, it returns the two regions. 
 If it finds samples that look like the seat, but cannot find the back
 above the seat, it continues looking for the seat. Finally, if it
 cannot find both regions, it returns NIL.
 For efficiency in searching the image, parts like the seat that have
 distinguishing attributes should be sought first, in order to narrow
 the search rapidly to likely areas of the scene.
 
 
 
 
 
 
 
 III.	Interactive Strategies
 
 	It is not always possible to specify a simple predicate that
 will select all image samples belonging to a desired object and to
 no others; in such cases, specialized object-finding programs may be
 needed to determine the actual boundaries of an object in the image.
 
 	The interactive features of our system provide unique
 experimental capabilities for rapidly developing specialized scene
 analysis procedures.  The system offers researchers the following
 advantages:
 
 	(1)   Complete compatibility with INTERLISP, facilitating
 	      the development and integration of specialized
 	      scene analysis algorithms.
 
 	(2)   Symbolic access to a large library of primitive
 	      feature extraction operators as well as to a wide 
 	      variety of utility programs (e.g., coordinate 
 	      transformations, display routines) underlying the
 	      implementation of the top level system.
 
 	(3)   An extensible vocabulary of semantic concepts that can
 	      be used to express perceptual strategies.
 
 	(4)   An interactive graphics package specifically designed
 	      for scene analysis experimentation.
 
 	(5)   An efficient and versatile iconic data structure
 	      for regions and edges, complete with editing and display
 	      routines.
 
 	The system has been used extensively 
 for constructing specialized programs that find and outline the various
 objects in a domain of uncluttered office scenes.  In this work, the
 system's basic repertoire of primitive operators was augmented to include
 several higher-level functions.  The following examples demonstrate
 the use of ISIS for interactively generating strategies and techniques 
 for finding several objects in room scenes. Later chapters will describe 
 the implementation of programs to emulate these techniques automatically. 
 
 
 Development of a Door-Finding Program
 
 	One might assume from previous examples that, to find the door,
 the system need only apply a suitable predicate to extract door samples
 in the image, and then fit a boundary if desired.  Door samples, however
 are not easily distinguished from those of other objects on the basis
 of local attributes.  This fact was established empirically by circling
 a door in a test image and requesting the system to select random image
 points that fell within the ranges of hue, saturation, height, and surface
 orientation observed in that region.  (Brightness was considered too
 variable to use as a criterion for acquisition.)  Figure 11 shows that
 many irrelevant points met the criteria, including points on the chair
 back, points in the picture hanging on the wall, and even points on
 the portion of wall visible under the table.  (The saturation of wall
 points under the table is unusually high because incident light is 
 reflected from the brown floors.  The saturation of some door points
 is reduced because of specular reflection.)
 
 	The vertical rectangle bounder (VRB) is a highly specialized
 program module that can utilize a priori knowledge to obtain an approximate
 door boundary rapidly, without recourse to edge-following or region-growing.
 All SRI office doors are known to be vertical, rectangular, plane
 surfaces, 3.5 feet in width, 7 feet high, and bounded below by the
 floor (height 0).  The boundary of a door in an image is thus completely
 constrained given the world coordinates of a single point on either
 vertical edge and the surface orientation measured on the door's side
 of the edge.*  The task of finding the door thus reduces to that of
 obtaining the two pieces of information, an edge location and a surface
 orientation required by the vertical rectangle bounder.
 
 ----------------------------------------------------------------------
 *Assuming the detected edge has world coodinates (X, Y, Z, the
 door boundary is a vertical rectangle with two corners at (X, Y, 0) and
 (X, Y, 7) and two other corners at the same heights but 3.5 feet
 horizontally displaced along the known plane of the door.
 ----------------------------------------------------------------------
 
 	In the current image (Figure 12), observe that the height range
 above the tabletop and below the picture is occupied only by the door
 and the directly illuminated wall.  These objects were observed to 
 differ markedly in saturation, suggesting that a door edge could be
 located by scanning horizontally at this distinguished height, seeking
 a saturation discontinuity.
 
 	The above conjecture was confirmed with some brief experimentation:
 First we obtained numerical values of saturation using the primitive
 SAT.  The saturation was observed to be 0.6 when pointing at a door
 sample and 0.25 when pointing at a wall sample.  We then used the
 scanned horizontally across
 the image in the specified height range seeking a saturation jump of
 at least 0.2 (the scan locus was manually designated by pointing at
 suitable end points on each side of the image).  The left to right
 scan, starting on a door sample successfully terminated at the right
 door boundary where the required saturation discontinuity was first
 detected.
 
 	To determine a suitable scan line automatically, we first
 determined the actual height of the bottom edge of the picture frame
 (2.8 feet) by pointing in the image and executing (HEIGHT CURSOR).
 We then scanned down from the top along each side edge
 of the image, seeking the first sample with height less than that
 of the picture bottom.  The two points found 
 determine a suitable horizontal locus on which to seek a
 saturation discontinuity.  The operation of this phase of the door-finding
 program is summarized in Figure 12a-c.
 
 	The world coordinates (X, Y, Z) of the resulting edge point were
 then obtained using the primitive function XYZ.  The surface orientation
 of the door was found by applying the primitive function SORQ to a door
 sample on the high saturation side of the discontinuity.  Finally, the
 door boundary was computed from this information by the vertical rectangle
 bounding module.
 
 	The computed boundary was passed to a display module for presentation
 superimposed on the original image.  Note that in the present scene the
 door extends above and to the left of the image borders.  Hence the
 display module must clip the actual computed boundary to coincide with
 the visible portion of the door.  Figure 12d shows the resulting boundary
 display.  The total development time for this strategy including several
 false starts was about four hours.  Strategies outlined below for finding 
 the wall, picture, table and chair involved comparable effort.  
 
 
 
 Some Illustrative Object Finding Strategies
 
 	
 	In this section, we briefly outline several additional object
 finding strategies developed empirically with our system.
 
 1.	Walls
 
 	The wall-finding strategy was developed by slightly modifying
 the door-finding strategy.  A step decrease in saturation while scanning
 horizontally left to right marks the left edge of the wall.  Unlike doors,
 however, the widths of wall sections are not known a priori.  Hence the
 scan must continue seeking a step increase in saturation that marks
 the right edge of the wall.  Given these two lateral extremes, and the
 known vertical extent, the VRB can infer the complete boundary of a 
 section of wall.
 
 2.	Pictures
 
 	Since the height and width of pictures are both variable
 parameters, the VRB requires at least one edge
 point on all four borders to compute the picture boundary.  The
 search for the border samples is constrained by the fact that pictures
 hang on walls.  Hence the first step in picture finding is to call the
 wall finder.  The program next scans in horizontally at nominal picture
 height from both the left and right wall edges, seeking discontinuities
 in hue or saturation.  These discontinuities mark the left and right
 vertical picture borders respectively (an extent check is made to 
 distinguish these edge points from those on the wall-door boundary).  
 The horizontal top and bottom
 picture borders are found by scanning vertically down from the top of
 the image and up from table height in between the previously detected
 lateral extremes of the picture, seeking color discontinuities.  The
 picture boundary is then a rectangle consisting of two vertical lines
 through the (X,Y) world coordinates of the left and right edge points,
 extending from the height (Z coordinate) of the bottom edge point to
 the height of the top edge point and the two horizontal lines connecting
 the tops and bottoms of these verticals.
 
 3.	Tabletop
 
 	The tabletop boundary is found using GROW to extend a region 
 outward from points located on the tabletop by the predicate
 (TABLETOP X).  The predicate used by the region grower is one composed
 of tests for points having attribute values within the extreme values
 for tabletop.
 
 
 
 IV   Discussion
 
 	The interactive style of programming encouraged by this system
 is in marked contrast to the awkwardness usually experienced in writing
 and debugging picture-processing routines; strategies that are trivial
 to describe to a human programmer invariably take weeks to translate
 into code using conventional programming systems.  We feel that high-level
 graphical interaction is an invaluable tool for developing scene analysis
 procedures.
 
 A.	Generality
 
 	The programs and representations discussed in this paper
 admittedly are ad hoc and highly context dependent; it is for this
 reason that they are so effective.  These programs are based on a
 type of domain-specific knowledge that is difficult to utilize in a
 general-purpose scene analysis system, but very easy for a person to
 supply interactively.
 
 	Such highly specialized strategies can be criticized for their
 lack of generality.  The door-finding program can be made to fail, for
 instance, by placing a tall object on the table and slightly shifting
 the point of view:  the basic philosophy followed, though, is not
 to complicate a strategy or representation unnecessarily, by anticipating
 all possible contingencies.  Should a program actually fail, we would
 then try to patch it by interactively experimenting with additional
 constraints.  For example, if the door-finding program became confused
 by the edge of a tabletop object we could easily install an additional
 test that would check the vertical extend of a discontinuity before
 calling the bounding module.  Alternatively, the door side of the
 discontinuity could be checked for proper hue, saturation, and surface
 orientation.  As a final check, a mask can
 be used to find discontinuities in the image that confirm the boundaries
 predicted by the bounding module.  It is as hard to guess which
 refinements will prove effective in overcoming a bug as it is to anticipate
 which bugs will materialize.  Fortunately, an interactive system allows
 refinements to be added empirically as contingencies arise.
 
 B.	Primitive Functions
 
 	The ability to describe concepts and strategies interactively
 in a particular domain depends of course on the availability of
 appropriate primitive functions for that domain.  In office scenes,
 there was heavy reliance on multisensory features (hue, surface
 orientation, height) obtained from color and range data to increase
 the likelihood of finding simple, distinguishing attributes for each
 object.  The above features would obviously be ineffectual for distinguishing
 objects in a black and white outdoor scene, where objects are distinguished
 principally by attributes such as shape and texture as well as by 
 spatial relationships.  We are currently implementing some additional
 primitive functions which will extract relevant features for representing
 shapes and textures.
 
 	The scene analysis literature abounds with descriptions of low-level
 operators *REF*.  Unfortunately it is very difficult to anticipate which
 set of operators will provide adequate discrimination in a given domain.
 Given the interactive nature of the approach, it is more reasonable
 to start with a few likely operators and let the set grow with
 necessity; i.e., when the available operators are unable to distinguish
 examples of a new concept from examples of previously defined concepts,
 then additional operators are introduced.  The ability to experiment
 interactively is an enormous help in selecting new operators and
 ultimately may help to establish a taxonomy of operator capabilities.
 
 C.	Distinguishing Features
 
 	Again, we wish to emphasize that our use of distinguishing features 
 to avoid details description should not be construed as a denigration 
 of other work on complex representations.  Assuredly, occasions will arise
 when, like a person, a machine will have to resort to detailed descriptions
 in order to discriminate among very similar objects.  Our main contention
 is that relatively simple representations suffice most of the time.
 The advantages of a simple description include economy, reliability, and
 most important, the ability to find complex objects (like trees)
 for which descriptively adequate representations are still unknown.
 Since all representations are incomplete at some level of detail, there
 should not be undue concern over using the simplest representation that
 provides adequate discrimination for the current task.
 
 D.	User Interface
 
 	During the design of the present system, considerable
 attention was paid to the man-machine interface.  We wanted a system
 that was easy to learrn and that would encourage people to experiment
 with a variety of descriptions and perceptual strategies.  The following
 design featues contribute to the operating simplicity of the system:
 
 	(1)   The principal power of the system resides in a handful
 	      of versatile top-level functions listed in Appendix 2.
 	      These functions respond appropriately to any meaningful
 	      argument.  For instance, an image sample, the cursor,
 	      a region, an object, and a defined color are all valid
 	      arguments to HUE.
 
 	(2)   All functions default interactively; if the experimenter
 	      uses an unknown concept to describe a new object, the
 	      system will request a definition (in terms of the
 	      attributes of some known object or by graphical example)
 	      before allowing him to continue.
 
 	(3)   Requests for input are stated concisely, often with a
 	      single prompt character, to spare the patience of 
 	      exprienced users.  An inexperienced user can always
 	      request an explicit list of options by responding with
 	      a "?".
 
 	(4)   The system maintains a continuity in the man-machine
 	      dialogue by interpreting underspecified commands in
 	      the context of recent processing.  For instance, after
 	      requesting the system to find all "green" regions, the
 	      experimenter need only enter the function (HEIGHT) to
 	      obtain a distribution of heights for all green areas
 	      of the scene.
 
 	Amenities such as these make it possible for many visitors to
 use the system after only a brief demonstration.
 
 E.	Incremental Growth
 
 	We view our system as a prototype facility for use in
 developing spcialized scene-analysis programs in many application
 areas.  The power of this facility can be expected to grow incrementally
 with use.  The more primitive operators that have been programmed, the
 more likely one is to find distinguishing representations for new
 semantic attributes.  Similarly, as more semantics are defined, it
 becomes easier to describe new objects.  Finally, a large repertoire
 of object-finding strategies will frequently contain useful components
 of new strategis.  In the office domain, a previously developed
 wall-finding program was utilized in a strategy for finding pictures
 that are constrained to hang on walls.  Judging from exprience in
 office scenes, when appropriate primitive operators semantics, and
 procedures have been accumulated for a domain, programming in that
 domain becomes easy.
 
 
 
 
 
 
 
 See you Friday.
 
 Tom
 -------